Transparent Reporting of Ethics for Generative AI: the TREGAI Checklist

The widespread use of Chat Generative Pre-trained Transformer (known as ChatGPT) and other emerging technology that is powered by generative artificial intelligence (GenAI) has drawn attention to the potential ethical issues they can cause, especially in high-stakes applications such as health care, but ethical discussions have not yet been translated into operationalisable solutions. Furthermore, ongoing ethical discussions often neglect other types of GenAI that have been used to synthesise data (eg, images) for research and practical purposes, which resolve some ethical issues and expose others.

We did a scoping review of the ethical discussions on GenAI in health care to comprehensively analyse gaps in the research. To reduce the gaps, we have developed a checklist for comprehensive assessment and evaluation of ethical discussions in GenAI research:

Ethical principles covered in the checklist are summarised below:

The latest checklist and definitions of ethical principles can be downloaded as a Word document: TREGAI checklist.docx.

We develop the TREGAI checklist for use by journals, institutional review boards, funders, regulators, and other potential users as a facilitating tool to promote systematic ethical investigations in GenAI research. As an immediate extension, the TREGAI checklist can be used for ethical assessments of GenAI-powered products derived from research based on user manuals.

Citation

Usage

This work is licensed under CC BY 4.0

Download the latest checklist as a Word document: TREGAI checklist.docx.

  • We strongly recommend reading this checklist in conjunction with the reference article above for important clarifications on all the items.
  • We will update the checklist when there are updated ethical principles, recommended actions, or progresses in generative AI regulations and guidelines.

The TREGAI checklist is provided to systematically document ethical assessments in development or applications of generative AI, specifically:

  • identification of ethical issues caused, or mitigation of existing issues,
  • any solutions to issues identified, or inadequacies of mitigation methods, and
  • explicit document relevant elaboration and justification.

The checklist is applicable for:

  • Generative AI research: to facilitate in-depth discussion during peer review and after publication.
  • Generative AI-powered products: to disclose ethics-related considerations and measures to help users establish reasonable trust.
  • Generative AI-created contents (e.g., on social media or in education): to disclose benefits, limitations and potential risks on relevant ethical aspects.

Worked example

To demonstrate the usage of TREPAI, we use it below to assess the ethical discussions in Singhal et al (2023) as an example. This paper developed Med-PaLM by adapting Flan-PaLM to the medical domain, and curated benchmark datasets of medical questions to evaluate Flan-PaLM, Med-PaLM and other large language models.

For ethical discussions, we indicated the position of manuscript text for dedicated discussions on relevant ethical principles, as well as details on the method used to assess ethical issues.

Language

English

Contact